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Chapter  1 


PROBLEM  STATEMENT 

To  the  acquisition  program  manager,  accurate  cost 
information  is  essential  for  effective  program  management. 
Congress,  and  the  public  in  general,  demand  accountability 
for  the  spending  of  tax  dollars  particularly  for  dollars 
spent  in  the  acquisition  of  complex  defense  systems.  At  the 
product  divisions  within  Air  Force  Systems  Command  (AFSC), 
the  program  manager  is  held  accountable  for  the  technical 
success  and  cost  of  an  acquisition  program.  He  is  the  key 
decision  maker.  Cost  information  must  be  provided  in  a  form 
and  at  a  time  useful  to  him  (1:33)* 

For  his  cost  information,  the  program  manager  relies 
on  a  cost  analyst.  The  cost  analyst  looks  at  the  cost,  per¬ 
formance  data  he  is  receiving  from  the  contractor  to  decide 
what  information  he  should  emphasize  and  how  the  information 
should  be  presented  to  the  program  manager.  Depending  on 
the  dollar  threshhold  and  type  of  contract,  the  contractor 
is  required  to  submit  either  a  monthly  cost  performance  re¬ 
port  (CPR)  or  cost/schedule  status  report  (C/3SR)  to  the  pro¬ 
gram  office.  The  contractor  provides  data  in  these  reports 
which?  indicates  work  in  progress,  relates  cost,  schedule  and 
technical  performance,  provides  valid,  timely,  and  auditable 
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training,  and  supplies  DOD  managers  with  a  practicable 
level  of  summarization  (2:2-1). 

A  critical  item  of  information  the  contract  cost 
analyst  supplies  to  the  program  manager  is  an  estimate  of 
the  contract  cost  at  completion  (CAC).  The  estimate  is  used 
to;  assess  the  program's  budget  and  its  phasing,  and  to 
forecast  cost  overruns  (or  underruns)  and  support  funding  re¬ 
quirements.  Cost  analysts  in  the  Air  Force  often  make  use 
of  one  of  two  cost  forecasting  models  to  generate  an  estimate 
at  completion  (EAC):  1)  the  Automated  Financial  Analysis 
Program,  a  User's  Guide,  Electronic  Systems  Division,  Nov¬ 
ember  1976;  or  2)  a  Cost  Performance  Forecasting  Concept  and 
Model,  Aeronautical  Systems  Division,  November  197^-  The 
purpose  of  this  thesis  is  to  compare  these  two  cost  fore¬ 
casting  models  by  noting  their  respective  accuracy  when  ap¬ 
plied  to  actual  observations  of  historical  acquisition  pro¬ 
grams  . 

Both  models  cited  above  utilize  monthly  cost  and 
schedule  data  supplied  by  the  contractor  in  one  of  two  re¬ 
ports;  the  Cost/Schedule  Status  Report  (C/SSR),  or  the  Cost 
Performance  Report  (CPR).  The  Cost/Schedule  Status  Report 
is  applicable  to  Air  Force  contracts  which  have  estimated 
Research,  Development,  Test,  and  Evaluation  (RDT&E)  costs 
between  $2  million  and  $25  million  or  which  have  estimated 
production  costs  of  between  $2  million  and  $100  million. 

The  Cost  Performance  Report  is  applicable  to  contracts 
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which  have  estimated  RDT&E  or  production  costs  greater  than 
$25  million  or  $100  million,  respectively.  These  re¬ 
ports  serve  as  the  basis  by  which  the  contract  cost  analyst 
can  assess  the  contractor’s  progress  toward  meeting  the 
schedule  and  cost  specified  in  the  contract. 

At  the  onset,  the  reader  must  clearly  recognize  that 
this  thesis  does  not  address  initial  weapon  system  cost  esti¬ 
mating  or  pricing.  This  thesis  deals  with  two  models  which 
forecast  or  estimate  a  contractor's  cost  position  at  the  com¬ 
pletion  of  the  contract.  To  remember  this  distinction,  it 
is  helpful  to  keep  in  mind  that  a  contract  has  already  been 
awarded  and  of  interest  is  the  contractor's  performance. 
Additionally,  the  reader  should  recognize  that  the  two  models 
under  discussion  in  essence  represent  mathematical  manipula¬ 
tions  of  historical  cost  performance  data.  This  approach  is 
but  one  pursued  by  the  contract  cost  analyst.  Cost  perfor-  • 
mance  is  also  estimated  using  technical  assessments  and  in¬ 
formation  provided  by  program  office  engineering  personnel. 

A  mathematical  approach  should  not  be  viewed  in  isolation 
but  rather  used  to  support  or  in  conjunction  with  other 
available  information. 

TERMS  AND  DEFINITIONS 

The  reader  requires  a  basic  understanding  of  the 
measures  provided  in  the  contractor's  monthly  C/SSR  or  CPR. 
Fortunately ,  these  measures  are  few  and  are  briefly  defined 
as  follows: 
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Budgeted  Cost  for  Work  Scheduled  (BCWS)  -  BCWS  is 
the  time-phased  budget  which  is  the  result  of  associating  a 
budget  for  each  task  a  contractor  will  perform  with  the  ap¬ 
propriate  schedule  for  that  task.  The  time -phased  budgets 
for  tasks  are  aggregated  yielding  a  plan  in  terms  of  dollars 
and  time  against  which  actual  task  accomplishment  can  be 
measured. 

Budgeted  Cost  for  Work  Performed  (BCWP)  -  BCWP  is 
the  sum  of  the  budgets  assigned  to  completed  work  plus  an 
assessment  of  in-progress  work.  BCWP  represents  all  work 
completed  from  the  beginning  of  the  contract  through  the  end 
of  the  current  accounting  period.  It  in  essence  represents 
the  physical  accomplishments  of  a  task,  in  which  case  BCWP 
for  a  completed  task  equals  BCWS  for  that  task.  For  in- 
process  work,  BCWP  is  derived  by  actual  measurement  or  an 
assessment  of  the  status  of  that  task.  For  example,  suppose 
$100  is  budgeted  to  accomplish  10  hours  of  an  engineering 
task.  If  the  task  has  been  accomplished,  BCWP  equals  $100. 

If  5  hours  of  the  task  has  been  accomplished  and  the  cost  ac¬ 
count  manager  estimates  the  task  is  indeed  50  percent  com¬ 
plete,  BCWP  for  the  in-process  work  is  $50. 

Actual  Cost  for  Work  Performed  (ACWP)  -  ACWP  re¬ 
presents  those  direct  and  indirect  costs  identified  specif¬ 
ically  to  the  contractual  efforts  being  reported.  It  re¬ 
presents  the  consumption  of  labor,  material,  and  other  re¬ 
sources.  Again  returning  to  the  simplistic  example  of  the 
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engineering  effort  cited  above,  if  the  engineering  task 
actually  required  11  hours  of  effort  at  $10  per  hour,  the 
ACWP  for  the  task  would  equal  $110. 

Before  presenting  two  additional  data  elements  im¬ 
portant  in  performance  report  analysis,  the  three  measures 
defined  above  are  viewed  together  using  the  simplistic  en¬ 
gineering  task.  Suppose  the  contract  consisted  solely  of 
one  task,  the  engineering  effort,  scheduled  to  be  accom¬ 
plished  by  time  t,  with  a  budgeted  cost  of  $100.  The  budg¬ 
eted  cos't  for  work  scheduled  at  time  t,  is  $100.  If  at 
time  t,  the  engineering  effort  is  incomplete,  however,  and 
an  assessment  is  that  75  percent  of  the  task  has  been  ac¬ 
complished,  then  BCWP  equals  $75*  Suppose  further  that  to 
accomplish  this  effort  the  contractor  has  encountered  tec*.  • 
nical  difficulties  and  accrued  actual  costs  of  $110,  then 
ACWP  at  time  t,  equals  $110. 

Budgeted  at  Completion  (BAC)  -  BAC  equals  the  total 
budget  for  a  particular  element  throughout  the  life  of  the 
program.  In  other  words,  BAC  equals  the  summation  of  BCWS 
for  the  element.  The  BAC  for  any  element  reported  in  the 
performance  report  consists  of  the  original  budget  for  the 
element  adjusxed  for  contractual  changes,  internal  replanning, 
assignment  of  undistributed  budget,  or  application  of  manage¬ 
ment  reserves. 

Latest  Re/ised  Estimate  (LRE)  -  The  LRE  is  the  cur¬ 
rent  estimate  of  costs  developed  by  the  contractor  based 
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on  past  performance  and  future  conditions  which  are  expected 
to  influence  the  contract.  The  LRE  should  be  revised  by  the 
contractor  as  required  to  provide  an  accurate  and  timely 
estimate  of  final  costs. 

The  five  data  elements  defined  above;  BCWS,  'BCWP, 
ACWP,  BAC,  and  LRE  are  the  crucial  elements  of  performance 
report  analysis.  To  be  sure,  additional  data  elements  are 
presented  throughout  this  analysis.  However,  a  definition 
and  explanation  of  construction  will  be  provided  when  a  new 
term  or  measure  is  used.  Having  defined  the  crucial  elements 
of  performance  report  analysis,  a  general  description  of  the 
ESD  and  the  ASD  models  is  provided. 

GENERAL  MODEL  DESCRIPTIONS 

The  ESD  cost  at  completion  (CAC)  forecasting  model  is 
actually  comprised  of  six  methods.  All  six  methods  calcul¬ 
ate  a  performance  factor  (PF)  based  on  a  relationship  be¬ 
tween  BCWP  and  ACWP.  The  performance  factor  is  multiplied 
by  the  budgeted  cost  of  work  remaining  (BCWR),  calculated  by 
subtracting  the  cumulative  BCWP  from  the  BAC,  to  generate  an 
estimated  cost  for  the  remaining  contractual  effort.  The 
estimate  for  remaining  work  is  added  to  cumulative  actual 
costs  to  data  to  provide  an  estimated  cost  at  completion. 
Specifically,  the  performance  factor  is  calculated  as 
follows: 

Performance  Factor  *  1.0  -  (3«A2-1) 


All  of  the  methods  are  linear,  however  two  of  methods  use 
performance  factors  determined  by  a  subjective  process. 
Although  not  included  in  this  analysis,  the  two  methods 
using  a  subjective  PP  are  valuable.  The  methods  allow  the 
program  office  to  adjust  the  PF  when  a  contractor's  past 
cost  performance  is  not  indicative  of  his  anticipated  future 
performance . 

The  ASD  cost  at  completion  forecasting  model  is  com¬ 
prised  of  two  estimating  methods.  Both  ASD  methods  are 
based  on  a  non  linear  relationship  between  BCWP  and  ACWP. 
Rather  than  a  straight  line  as  in  the  ESD  model,  the  long 
term  growth  relationship  in  the  ASD  model  is  represented  by 
a  curve -linear  form.  The  assumed  relationship  is: 

ACWP  =  bx  BCWP  4 

where: 

b^  and  b2  represent  the  growth  parameters  of  the  re¬ 
lationship. 

In  one  ASD  method,  monthly  CPR  or  C/SSR  observations  of  ACWP 

and  BCWP  are  used  to  compute  both  parameters,  b^  and  b2. 

This  method  is  referred  to  as  the  "unconstrained  exponential 

method”.  A  cost  at  completion  estimate  is  calculated  by 

substituting  the  current  BAC  in  the  equation.  In  the  second 

ASD  method,  a  value  is  assumed  for  b9.  After  estimating  the 

remaining  parameter  b^,  a  forecast  is  calculated  following 

the  same  procedure  described  above.  The  second  ASD  method 

is  known  as  the  "constrained  exponential  method"  (4:19). 
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The  potential  utility  of  the  "constrained"  exponential  method 
is  explained  in  cost  research  report  No.  117; 

.  .  .  if  a  large  family  of  completed  program 
samples  are  simila^  in  their  [growth]  characteris¬ 
tics  and  y  =  b.  x  ^  is  a  reasonable  behavioral 
relationship  tnen:  (a)  one  may  use  the  relation¬ 
ship  to  extrapolate  an  EAC  after  each  CPR  is  sub¬ 
mitted  throughout  the  program  life.  (b)  It  is 
reasonable  to  expect  that  the  computed  parameters 
b^  and  b2  would  reflect  these  similarities.  .  .[4:15]. 


OBJECTIVES 


A  number  of  terms  are  defined  in  proceding  para¬ 
graphs  and  subsequent  chapters  which  may  be  confusing  to  an 
individual  uninitiated  in  contract  cost  analysis.  A  general 
overview  of  the  use  of  the  terms  places  them  in  context.  A 
contractor  plans  his  effort  associating  budgets  to  tasks 
over  time.  This  budget  expressed  over  time  is  BCWS,  his 
schedule.  The  budget  for  a  task  irrespective  of  time  is  in 
essence  BCWP,  the  planned  cost  of  the  task.  Actual  costs  are 
accumulated  and  compared  to  the  planned  budget.  If  actual 
cost  for  an  effort  exceeds  planned  cost,  the  contractor  is  in 
an  overrun  position.  If  actual  cost  is  less  than  planned 
cost,  the  contractor  is  in  an  underrun  position. 

This  study  intends  to  achieve  three  specific  objec¬ 
tives  stated  as  follows; 

1)  Determine  which  of  the  two  forecasting  models  is 
more  accurate  in  forecasting  the  cost  at  completion  of  a 
contract. 
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2)  Determine  for  aircraft  acquisition  programs  para¬ 
meter  values  of  bg  for  use  in  the  "constrained"  cost  per¬ 
formance  forecasting  method. 

3)  Determine  which  of  the  two  forecasting  models 
is  more  accurate  when  the  contract  is;  less  than  25  percent 
complete,  greater  than  25  "but  less  than  50  percent  complete, 
greater  than  50  percent  hut  less  than  75  percent,  and  greater 
than  75  percent  complete. 

Preliminary  analysis  of  the  two  forecasting  models 
reveals  fundamental  mathematical  differences  in  their  con¬ 
struction.  Linear  percentage  cost  variance  extrapolation, 
methods  apply  the  current  or  adjusted  cost  variance  percent¬ 
age  to  the  budgeted  cost  of  work  remaining  to  generate  an 
estimate  at  completion.  The  exponential  methods,  on  the. 
other  hand,  are  based  on  an  underlying  non-linear  performance 
relationship. 

First  Hypothesis 

The  exponential  methods  are  more  accurate  than  the 
linear  cost  variance  extrapolation  methods  in  forecasting 
an  estimate  at  completion. 

Second  Hypothesis 

The  exponential  methods  are  more  accurate  than  the 
linear  cost  variance  extrapolation  methods  in  forecasting 
estimates  at  completion  when  a  program  is;  less  than  25 
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percent  complete,  greater  than  25  percent  but  less  than  50 
percent  complete,  and  greater  than  50  but  less  than  75  per¬ 
cent  complete. 

Third  Hypothesis 

When  estimates  are  generated  for  a  program  which  is 
greater  than  75  percent  complete,  one  model  is  not  signific¬ 
antly  more  accurate  than  the  other.  As  a  program  approaches 
completion,  the  models  yield  the  same  results. 

'W 

* 

Fourth  Hypothesis 

If  a  sample  of  completed  aircraft  acquisition  pro¬ 
grams  are  similar  in  their  characteristics,  the  parameters  b^ 
and  b2  reflect  these  similarities.  A  premise  upon  which  the 
ASD  model  hinges  is  that  the  extreme  values  of  the  exponent 
b^  is  narrow,  between  1.18  and  .97,  and  in  the  majority  of 
cases  b2  falls  between  1.10  and  1.0  (4: 15)* 

Prepared  by  the  contract  cost  analyst,  an  estimate 
of  cost  at  completion  is  a  useful  item  of  information  to  the 
program  manager.  The  estimate  is  used  to  assess  the  program 
budget  and  support  funding  requirements.  General  descrip¬ 
tions  of  two  cost  forecasting  models  are  provided  noting  the 
mathematical  differences  in  estimate  calculation.  Chapter  2 
is  a  review  of  literature  providing  additional  information 
regarding  the  two  models  under  analysis  and  descriptions  of 
other  models  which  are  used  in  cost  ‘forecasting. 
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Chapter  2 


LITERATURE  REVIEW 

The  review  of  applicable  literature  is  concentrated 
in  three  areas;  literature  descriptive  of  the  two  models 
under  analysis,  basic  reference  materials  which  explain  the 
mathematical  concepts  upon  which  the  models  are  based,  and 
reference  material  describing  other  cost  forecasting  models 
used  to  estimate  a  CAC. 

The  Automated  Financial  Analysis  Program,  a  User's 
Guide,  published  oy  the  Cost  Analysis  Division,  Comptroller, 
Electronic  Systems  Division,  November  1976,  provides  the 
basic  description  and  algorithms  of  six  methods  of  forecasting 
a  Cost  at  Completion  (CAC).  The  methods  of  computing  a  cost 
at  completion  are  based  on  the  assumption  that  the  computed 
cumulative  percent  cost  variance 

(  BCWP  -  ACWP  v  1nn\ 

{  — bUwp - x  loo; 

may  be  linearly  extrapolated  to  the  end  of  a  program  as  an 
estimate  of  final  cost. 

The  AFSC  Guide  for  the  Use  of  Contractor-Reported 
Cost  Data,  dated  1  July  1972  examines  and  describes  the  ap¬ 
plication  of  the  percentage  cost  variance  extrapolation  and 
cumulative  performance  index  techniques  to  generate  an 
Estimate  at  Completion  (EAC).  The  Cumulative  Performance 
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Index  (CPI  =  BCWP/ACWP)  is  applied  to  the  BAC  using  the  equa¬ 
tion,  EAC  =  BAC/CPI.  Results  generated  by  either  procedure 
are  identical.  A  shortcoming  noted  regarding  the  techniques 
is  their  failure  to  account  for  the  overall  completion  status 
of  a  program,  i.e,  whether  the  program  is  5  percent  or 
95  percent  completed.  Estimates  generated  with  these  tech¬ 
niques  near  program  completion  tend  to  be  quite  accurate, 
while  estimates  generated  in  the  early  stages  of  the  contract 
may  be  misleading  (2:A-6). 

Cost  Research  Report  No.  117*  a  Cost  Performance  Pore 
casting  Concept  and  Model,  published  by  the  Cost  Analysis 
Division,  Comptroller,  Aeronautical  Systems  Division,  Novem¬ 
ber  197^1  provides  the  description  of  two  methods  of  develop¬ 
ing  an  exponential  curve -linear  form  for  forecasting  an  esti 

mate  at  completion.  The  exponential  relationship  chosen  is 

bo 

of  the  form  y  =  b^  x  with  the  parameters  b^  and  b2  esti¬ 
mated  by  using  a  non-linear  least  squares  technique.  The 

"unconstrained"  technique  uses  BCWP  and  ACWP  values  given  to 

b2 

date  in  the  function  ACWP  =  b^  BCWP  .  Both  b^  and  b2  are 
calculated  using  a  least  squares  technique.  In  turn,  the 
computed  parameters  and  current  BAC  are  used  to  compute  an 

b2 

estimate  at  completion  given  by  the  function  EAC  =  b^  BAC 
The  "constrained"  technique  specifies  a  value  for  the  para¬ 
meter  b2  based  on  historical  data.  After  solving  for  the 
parameter  b^,  an  EAC  is  generated  in  the  same  manner  as 
with  the  hnc ons trained " techni que .  It  is  suggested  in  the 
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ASD  report  that  the  collection  of  CPR  data  from  many  his¬ 
torical  programs  and  then  estimating  the  parameters  by  type 
of  acquisition  could  improve  the  accuracy  of  the  "constrained" 
forecasting  technique  (4:29). 

The  ESD  and  ASD  models  selected  for  comparison  in 
this  study  are  but  two  of  a  number  of  forecasting  models 
available  for  the  cost  analyst  to  use.  This  study  would  be 
remiss  without  mentioning  some  of  the  other  models  which  are 
available . 

A  model  developed  and  used  by  the  Army  is  the  Time 
Series  Analysis  for  Army  Internal  Systems  Management  (TSARISM). 
A  time  series  is  a  collection  of  observations  on  a  character¬ 
istic  generated  sequentially  in  time.  The  objective  of  the 
time  series  analysis  is  to  discover  the  relationship  between 
variables  and  for  each  variable  describe  its  behavior  over 
time.  The  TSARISM  program  identifies  a  generalized  model 
which  may  be  either  an  autoregressive  model,  a  moving  average 
model,  a  mixed  autoregressive -moving  average  model,  or  an 
integrated  form  of  all  three  models.  The  procedure  followed 
with  the  program  is;  1)  identify  the  possible  models,  2) 
select  a  model  and  estimate  the  parameters,  3)  form  the  fore¬ 
casting  equation,  4)  test  the  model  for  adequacy  through  re¬ 
sidual  analysis,  and  5)  forecast  with  the  model  (5)-  TSARISM 
is  a  direct  application  of  the  Box-JenKins  time  series  meth¬ 
odology  (6).  As  the  above  explanation  indicates,  the  TSARISM 
program  is  quite  complicated.  Further,  the  model  requires 


13 


approximately  twelve  or  more  observations  to  estimate  the  pa¬ 
rameters  and  evaluate  the  fitted  model.  Use  of  the  TSARISM 
program  basically  implies  a  dedicated  program  analyst  due  to 
its  complexity.  We  are  studying  the  ESD  and  ASD  models  bas¬ 
ically  because  of  their  simplicity  and  ease  of  application. 

A  model  proposed  by  Daniel  E.  Busse  is  identical  in 
functional  form  to  the  ASD  model  where  ACWP  =  z  (BCWP)e.  In 
this  model  e  is  a  sensitivity  factor  calculated  as  the  change 
in  ACWP  over  cumulative  ACWP  divided  by  the  change  in  BCWP 
over  cumulative  BCWP.  In  essence,  the  relationship  says  that 
actual  program  cost  as  a  function  of  planned  cost  is  propor¬ 
tional  to  the  current  ratio  of  these  variables  (7:25).  The 
proportionality  constant  is  the  sensitivity  factor  e  and  z  is 
an  offset  coefficient.  The  difference  between  this  model  and 
the  ASD  model  is  that  one  month's  data  is  used  with  this 
model  to  calculate  e  whereas  the  ASD  model  uses  a  least  mean 
squares  technique  applied  to  a  number  of  observations.  In 
the  Busse  model,  e  is  calculated  as  follows: 

-  ACWP/ACWPoum 

9  -  BCWP/BOWPcuffi 

Once  the  sensitivity  factor  e  is  determined,  the  offset  co¬ 
efficient  z  is  calculated  using  the  following  equation; 


The  entire  process  is  summarized  as  follows: 

1.  Use  the  information  in  the  latest  Cost  Perform¬ 
ance  Report  to  determine  the  parameters  e  and  z . 

2.  Use  the  latest  revised  BAC  furnished  by  the  con¬ 
tractor  in  the  functional  relationship  to  estimate  final  cost 

3*  ■  Repeat  the  process  each  month  as  new  cost  per¬ 
formance  data  is  received. 

The  appropriateness  of  the  model  is  questionable 
since  the  sensitivity  factor,  5,  may  fluctuate  widely  as  the 
program  evolves.  The  forecasting  procedure  gives  an  indica¬ 
tion  cf  final  cost  only  if  the  current  existing  cost  per¬ 
formance  trend  continues.  One  similarity  of  particular  im¬ 
portance  in  regard  to  this  model  and  the  ASD  and  ESD  models 
is  that  they  make  use  of  data  contained  in  Cost  Performance 
Reports  without  requiring  a  subjective  performance  factor. 

A  model  which  does  require  a  subjective  performance  factor 
is  discussed  in  the  next  paragraph. 

One  model  which  incorporates  a  subjective  perform¬ 
ance  factor  is  a  model  presented  by  J.  B.  Holeman  Jr., 
titled  "A  Product  Improved  Method  for  Developing  a  Program 
Management  Office  Estimated  Cost  at  Completion".  The  basic 
formula  of  the  model  is  as  follows: 

EAC  =  ACWP.,,_  +  ( BCWR  X  PPF) 
cum 


where : 


EAC  is  the  Estimated  Cost  at  Completion 

ACWP„,,„  is  the  total  Actual  Cost  for  Work  Performed 
cum 

to  date 

BCWR  is  the  Budget  Cost  of  Work  Remaining 
PPF  is  the  Predicted  Performance  Factor  or  a  pre¬ 
diction  of  what  it  will  actually  cost  for  each  $1.00 
worth  of  planned  work  remaining  (8s 22). 

The  model  actually  consists  of  three  options  for  determining 
a  predictive  performance  factor.  Option  one  uses  a  cost 
performance  index  (CPI).  This  cost  performance  index  is 
calculated  as  cumulative  ACWP  divided  "by  cumulative  BCWP. 

The  option  is  similar  to  the  performance  factor  used  in  the 
ESD  model.  The  CPI  does  not  explicitly  take  into  considera¬ 
tion  a  number  of  important  factors  which  could  affect  the 
EAC,  such  as  schedule  variations,  inflation,  and  technical 
risks.  Despite  these  shortcomings,  the  simplicity  of  the  CPI 
makes  its  use  attractive. 

The  second  option  to  determine  a  PPF  is  based  on 
five  factors  which  contribute  significantly  to  cost  change: 
changes  in  requirements,  inflation,  schedule  variations,  over¬ 
head,  and  unexpected  technical  problems.  Schedule  and  re¬ 
quirement  changes  are  incorporated  into  the  model  by  in¬ 
creasing  BCWR.  The  effects  of  inflation,  overhead  rates, 
and  technical  problems  are  reflected  in  the  performance  fac¬ 
tor.  This  option  is  particularly  useful  given  knowledge  of 
the  expected  rate  of  the  inflation  and  the  change  in  over¬ 
head  rates. 
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The  third  option  to  determine  a  PPF  is  basically  an 
extension  of  the  first  two.  Instead  of  determining  a  single 
value  for  the  PPF,  a  range  is  specified.  This  option  is 
used  at  the  lowest  level  of  the  Work  Breakdown  Structure  (WBS) 
consistent  with  the  practicality  of  getting  the  needed  in¬ 
formation.  Estimated  Costs  at  Completion  ranges  for  lower 
WBS  items  are  summed  to  the  aggregate  level.  The  second  and 
third  options  can  be  very  useful  when  knowledge  of  the  five 
factors  discussed  above  is  available  to  the  cost  analyst. 

The  final  model  reviewed  is  documented  in  the  Space 
and  Missile  Systems  Organization  (SAMSO)  Cost  Performance 
Forecasting  Study  dated  June  1977 •  The  model  is  actually 
comprised  of  two  methods.  One  method  uses  a  six  month 
straight  line  moving  average  projection  of  ACWP  and  BCWP 
to  generate  an  EAC.  BCWP  data  for  six  months  is  used  to  fit 
a  straight  line  via  least  squares  regression.  The  BCWP  fit¬ 
ted  regression  line  is  then  projected  to  a  point  in  time 
where  it  equals  current  BAG.  In  turn,  six  months  of  ACWP 
data  is  used  to  fit  a  straight  line  which  is  projected  to  the 
time  period  in  which  BCWP  equals  BAG.  The  result  is  an  esti¬ 
mate  of  the  contract  cost  at  completion  (9s8). 

The  second  method  uses  an  equation  that  more  closely 
fits  the  expenditure  data  than  a  straight  line. 

The  curve  that  showed  the  best  results  wa3  a 
modified  Erlang  equation*. 

y  =  axbecx  $  =  ACWP,  BCWP,  ECWS 
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e 


x  =  month 


$  =  Bx 


Then  the  ASD  model  was  used  to  provide  a  "better 
non-linear  least  squares  regression  fit  of  the  cum 
of  this  equation. 

y  =  5  a  x  be  cx  [9:21] 

x=0 


The  monthly  cumul xtive  BCWP  observations  are  used  to  fit  the 
curve  form  which  is  projected  as  described  above  to  a  point 
in  time  where  BCWP  equals  BAC.  Similarly,  monthly  ACWP  is 
fitted  with  a  curve  form.  As  with  the  six  month  straight 
line  moving  average  projection  method,  the  second  method 
generates  an  EAC  by  projecting  the  fitted  ACWP  curve  form  to 
the  point  in  time  where  BCWP  equals  BAC. 

Indeed,  all  of  the  above  models  are  useful  and  each 
may  be  particularly  appropriate  when  applied  to  a  specific 
type  of  program  (e.g.,  missile  procurement).  The  ASD  and 
ESD  models  are  chosen  for  their  simplicity  and  ease  of  ap¬ 
plication.  The  cost  variance  percentage  extrapolation  model 
(ESD)  is  intuitively  straight  forward.  The  ASD  model  is  log- 
linear  for  easy  computation  of  the  parameters  but  is  a  curve 
form.  The  ESD  model  is  compared  with  the  ASD  model  because 
the  assumed  functional  relationship  between  ACWP  and  BCWP  in 
the  ASD  model  is  non  linear.  A  straight  line  relationship 
between  cumulative  ACWP  and  BCWP  does  not  seem  to  represent 
the  long  term  growth  relationship. 
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Chapter  3 


METHODOLOGY 


Overview 

Chapter  3  consists  of  five  sections.  The  first  two 
sections  describe  the  population  of  interest  and  the  data 
collection  plan.  The  raw  data  elements  are  discussed  noting 
how  the  elements  are  collected  and  from  where.  Section  3 
presents  the  computational  formulas  used  in  each  method  to 
forecast  an  estimate  at  completion.  Four  ESD  methods  and 
the  two  ASD  methods  are  explained.  In  section  4,  a  measure 
of  accuracy  is  defined  as  well  as  its  calculation.  The 
fifth  section  explains  the  testing  procedure.  The  test  de¬ 
sign  is  related  to  the  objectives  and  hypotheses  of  the 
comparative  analysis.  To  test  for  the  equality  of  treatment 
means  a  single  factor  analysis  of  variance  (ANOVA)  is  used 
with  a  randomized  complete  block  design.  ANOVA  indicates  if 
there  are  differences  between  treatments,  but  exactly  which 
treatments  differ  is  not  specified.  To  determine  which  treat¬ 
ments  differ  Fisher’s  least  significant  difference  test  is 
applied  between  treatment  means. 

Sam-ple/Population 

The  universe  is  defined  as  all  DOD  procurement  pro¬ 
grams  which  require  contractor  submittal  of  either  a  cost/ 
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schedule  status  report  (C/SSR)  or  cost  performance  report 
(CPR) .  In  essence,  these  reports  are  applicable  to  RDT&E 
contracts  and/or  production  contracts  equal  to  or  greater 
than  $2  million  in  value.  The  populations  of  interest  con¬ 
sist  of  aircraft  and  avionics  RDT&E  and  procurement  con¬ 
tracts.  The  measurement  variables  collected  from  either  of 
the  two  reports  are  of  the  ratio  scale.  The  importance  of 
the  scale  cannot  be  overlooked  as  geometric  and  harmonic 
means  can  be  used  as  measures  of  central  tendency. 

A  random  sample  consisting  of  25  programs  is  drawn 
from  the  completed  program  contracts  library  at  Aeronautical 
Systems  Division.  All  contracts  in  the  sample  have  periods 
of  performance  greater  than  12  months.  Five  additional  air¬ 
craft  programs  are  chosen  and  added  to  the  sample  for  meas¬ 
uring  the  increase  in  accuracy  of  using  a  specified  value 
•of  b2  with  the  "constrained"  estimating  method.  At  best 
the  results  of  this  study  can  be  generalized  to  other  air¬ 
craft  and  avionics  contracts.  The  parameters  calculated 
based  on  the  sample  for  use  in  the  "constrained"  method  are 
applicable  only  to  similar  aircraft  procurement  programs. 

The  study  indicates  however,  whether  or  not  the  process  of 
calculating  parameters  for  use  with  the  constrained  esti¬ 
mating  technique  is  useful  in  increasing  the  accuracy  of 
estimates.  If  accuracy  is  improved,  the  procedure  could  be 
applied  to  other  contract  types  (e.g.,  software  and  missiles). 


20 


Data  Collection  Plan 


The  data  for  this  study  is  collected  from  cost  per¬ 
formance  reports  (CPR's)  and  cost/schedule  status  reports 
(C/SSR's)  on  file  in  the  program  contracts  library  at  Aero¬ 
nautical  Systems  Division.  Monthly  CPR  and  C/SSR  data  is 
filed  in  the  library  for  contracts  requiring  its  submittal. 
From  the  ASD  library  completed  contracts  are  selected  for 
aircraft  procurements  and  avionics  contracts  respectively, 
with  periods  of  performance  greater  than  12  months.  Com¬ 
pleted  programs  are  selected  because  the  last  CPR  submitted 
contains,  by  definition,  the  final  cost  for  the  contract. 

It  is  important  to  note  that  final  cost  is  being  addressed 
as  distinct  from  the  final  price  which  may  include  fee. 

For  each  contract  for  each  month,  the  following 
monthly  cumulative  data  elements  are  transcribed  from  the 
CPR's  to  a  columnar  working  sheet;  BCWS  -  the  budgeted  cost 
of  work  scheduled,  BCWP  -  the  budgeted  or  planned  cost  for 
the  work  actually  performed,  ACW?  -  the  actual  cost  of  work 
performed,  BAC  -  the  budget  at  completion,  that  is,  the  plan¬ 
ned  value  of  effort  to  be  accomplished  by  the  contractor, 
and  LRE  -  the  contractor’s  latest  revised  estimate  of  the 
cost  to  perform  the  contracted  effort.  A  separate  working 
sheet  is  used  for  each  contract.  The  format  for  the  working 
sheets  is  presented  in  Appendix  A. 

Each  worksheet  contains  the  cumulative  monthly  data 
elements  described  above  from  the  beginning  to  the  end  of 


21 


the  contract.  For  each  contract,  the  monthly  data  elements 
are  used  to  generate  an  estimate  at  completion.  Estimates 
are  generated  beginning  with  the  sixth  month  for  all 
methods.  Although  several  procedures  can  generate  esti¬ 
mates  with  as  little  as  three  months  of  data,  the  "uncon¬ 
strained"  procedure  in  the  ASD  model  requires  at  least  six 
months  of  data  for  estimating  the  parameters  by  the  least 
squares  technique.  The  two  estimating  models  are  applied  to 
each  contract.  The  individual  methods  comprising  each  model 
with  their  computational  formulas  are  discussed  in  the  next 
section. 

Methods  of  Estimating  a  Cost 
at  Completion  -  ESP  Model 

The  ESD  Automated  Financial  Analysis  Model  is  actu¬ 
ally  comprised  of  six  methods  which  can  be  used  to  generate 
a  cost  at  completion  (CAC).  In  this  study,  four  of  those 
methods  are  employed  which  do  not  require  a  subjective  per¬ 
formance  factor.  Often,  a  dedicated  program  analyst  pos- 
esses  knowledge  of  how  future  performance  is  anticipated  to 
vary  from  past  performance .  The  analyst  can  evaluate  and 
allow  for  potential  future  changes  in  performance .  The  use 
of  a  subjective  performance  factor  is  dependent  upon  the 
analyst's  knowledge  of  known  technical  risks  yet  to  come 
which  may  not  be  reflected  in  past  performance.  Obviously, 
the  insight  required  to  make  use  of  a  subjective  performanc 
factor  is  not  available.  As  such,  two  of  the  ESD  methods 
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are  omitted  from  consideration  in  this  study.  The  four 
methods  drawn  from  the  ESD  model  are  presented  with  step  hy 
step  instructions  on  their  calculation. 

Method  1 .  This  method  is  calculated  using  a  per¬ 
formance  factor  based  on  current  month  cost  performance  (3* 
A2-1).  It  is  calculated  as  follows: 

1.  Compute  current  month  BCWP  (BCWP  ) 


BCWP,,,,  =  cumulative  BCWP  (months)  - 
cr  t 

cumulative  BCWP  ( t -1 ) 

2.  Compute  current  month  ACWP  (ACWPcr) 

ACWPcr  =  cumulative  ACWP  (month^)  - 
cumulative  ACWP  (t-1) 

3.  Compute  the  performance  factor  (PP) 


PF  =  1.0  - 


BCWPOT,  -  ACWP_„ 
cr  cr 


BCWP. 


cr 


4.  Compute  quantity  of  work  remaining  (BCWR) 
BCWR  =  BAC  -  cumulative  BCWP 

5.  Compute  the  estimate  to  complete  (ETC) 

ETC  =  PF  X  BCWR 

6.  Compute  the  cost  at  completion  (CAC) 

CAC  =  ETC  +  Cumulative  ACWP 
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Method  2.  This  method  is  based  on  a  three  month 
moving  average  of  current  month  cost  performance  to  calculate 
the  performance  factor  (3*A2-2).  It  is  calculated  as  follows 

1 .  Compute  current  month  BCWPcr 

BCWP.,,  =  cumulative  BCWP  (month,  )  - 
cr  o 

cumulative  BCWP  (month^._^) 

2.  Compute  current  month  ACWPcr 
ACWPcr  =  cumulative  ACWP  (month^)  - 

cumulative  ACWP  (month^^) 

3.  Compute  current  BCWP  for  first  prior  month 

(BCWPor-l> 

BCWPcr_1  =  CUM  BCWP  (montht_1)  - 
CUM  BCWP  (montht_2) 

4.  Compute  current  ACWP  for  first  prior  month. 
(ACWPcr_1) 

I 

ACWPcr_1  =  CUM  ACWP  (montht_1)  - 
CUM  ACWP  (montht_2) 

5.  Compute  current  BCWP  for  second  prior  month 

(BCWPcr_2) 

BCWPcr _2  =  CUM  BCWP  (montht-2^  " 

CUM  BCWP  (montht_3) 
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6.  Compute  current  ACWP  for  second  prior  month 


8.  Compute  performance  factor 

1  (CV1  +  CV9  +  CV,) 

PF  =1.0' - 1 - - - 2- 

3 

9.  Compute  quantity  of  work  remaining  (BCWR) 
BCWR  =  BAC  -  CUM  BCWP 

10.  Compute  estimate  to  complete  (ETC) 

ETC  =  PF  x  BCWR 

11.  Compute  cost  at  completion  (CAC) 

CAC  =  ETC  +  CUM  ACWP 

cr 
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Method  3.  Method  3  is  calculated  using  a  perform¬ 
ance  factor  based  on  cumulative  cost  performance  (3:A2-4). 
Step  by  step,  it  is  calculated  as  follows: 

1 .  Compute  the  performance  factor  ( PF) 

CUM  BCWP(  month, )  -  CUM  ACWP(month.) 

PF  =  1.0 - - - — 

CUM  BCWP(montht) 

2.  Compute  quantity  of  work  remaining  (BCWR) 

BCWR  =  BAC  -  CUM  BCWP 

3.  Compute  the  estimate  to  complete  (ETC) 

ETC  =  PF  x  BCWR 

4.  Compute  the  cost  at  completion  (CAC) 

CAC  =  ETC  +  CUM  ACWP 

Method  4.  This  method  is  based  on  a  performance 
factor  calculated .by  using  the  last  quarter  cost  perform¬ 
ance  (3:A2-10).  Step  by  step,  it  is  calculated  as  follows 

1.  Compute  current  BCWPcr  (month^) 

2.  Compute  current  ACWPcr  (month^) 

3.  Compute  first  prior  BCWP{,r_1  (month^_1) 

4.  Compute  first  prior  ACWPcr_^  (month^._1) 

5.  Compute  second  prior  BCWPcr_2  (month^g) 

6.  Compute  second  prior  ACWPcr_2  (month^g) 


26 


last  CPR  while  method  4  is  an  adaptation  where  BCWP  and 
ACWP  are  summed  over  the  last  three  months  to  calculate  the 
performance  factor.  The  ASD  model  of  estimating  the  cost  a 
completion  for  a  contract  differs  fundamentally  from  the  ESD 
model  and  is  explained  in  the  next  section. 

Methods  of  Estimating  a  Cost 
at  Completion  -  ASP  Model 

The  ASD  cost  performance  forecasting  model  is  com¬ 
prised  of  two  methods?  an  "unconstrained"  exponential  method 
and  a  "constrained"  exponential  method.  The  "unconstrained" 
exponential  method  (Method  5)  makes  use  of  a  least  squares 
regression  analysis  technique  for  developing  trend  character- 
istics  and  the  assumed  relationship,  y  =  b^  .  x  .  The  rp- 
lationship  w"-r.-  'elected  for  the  following  properties; 

a.  1.  ost  sample;3  a  straight  line  relationship 
between  cumulative  ACWP  and  cumulative  BCWP  does  not  re¬ 
present  the  long  terra  growth  relationship  and  so  a  curve 
linear  relationship  is  considered  more  realistic. 

b.  The  curve  linear-  form  y  =  b^  .  x  can  easily 
be  compared  to  the  case  where  the  contractor  is  not  expe¬ 
riencing  a  cost  growth.  When  the  parameters  b^  and  b2  both 
equal  one,  y  =  x.  When  b2  is  greater  than  one,  y  has  a 
tendency  to  increase  faster  than  x  which  produces  an  ac¬ 
celerating  growth  curve; 

c.  The  exponential  functional  form  is  easily  trans¬ 
formed  to  a  linear  relationship  by  taking  the  natural 
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logarithms  of  the  function.  Thus  talcing  the  logarithms  of 
the  functional  form  ACWP  =  b^  BCWP  transforms  the  '  stion 
to  the  linear  form  LOG  ACWP  =  LOG  b1  +  b£  LOG  BCWP. 
transformed  relationship  allows  application  of  the  least 
squares  technique  to  compute  the  values  of  b^  and  b2  (4:13- 
14) .  A  step  by  step  summary  of  the  computational  process  is 
described  as  follows: 

Method  5.  1 .  Given  the  sample  of  monthly  cumula¬ 

tive  values  available  to  data  for  ACWP  and  BCWP  and  the  func- 

b2 

tional  form  ACWP  =  b^  .  BCWP  ,  the  function  is  transformed 
to  a  linear  relationship  LOG  ACWP  =  LOG  b.  +  b2  LOG  BCWP. 

The  least  squares  technique  computes  the  values  of  the  para¬ 
meters  b^  and  b2< 

.  2.  Using  the  current  value  of  the  budget  at  com¬ 

pletion  (BAC)  and  the, computed  values  of  b^  and  b2,  an 

estimate  at  completion  or  cost  at  completion  is  calculated 

b2 

using  the  function  EAC  =  b^  .  BAC 

3.  Steps  1  and  2  are  repeated  with  the  addition  of 
subsequent  monthly  CPR  data  (4»l6). 

The  important  point  to  note  in  the  use  of  the  "un¬ 
constrained"  method  is  that  the  parameters  are  recalculated 
as  additional  observations  are  available.  Both  parameters 
b^  and  b2  are  free  to  fluctuate.  The  "constrained"  method 
(Method  6)  is  constrained  in  that  a  value  is  specified  for 
the  parameter  b2.  The  parameters  calculated  from  a  few 
early  samples  are  inadequate  estimates  of  the  parameters 
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that  will  he  obtained  after  the  entire  population  of  data 
elements  for  a  given  contract  are  available .  The  "con¬ 
strained"  method  is  an  attempt  to  reduce  the  error  of  small 
sample  sizes  by  substituting  a  subjective  value  for  bg. 

It  was  noted  that  diverse  completed  samples  produced  sets 
of  parameters  in  which  the  extremes  of  the  exponent,  b2, 
was  narrow.  In  essence,  the  "constrained"  method  attempts 
to  reduce  the  error  in  the  parameters  by  placing  the  error 
in  a  historical  context  (4*19).  In  regard  to  this  study,  20 
historical  aircraft  procurement  programs  are  used  to  calcul¬ 
ate  bg  parameters  for  aircraft  procurements .  The  calculated 
parameters  are  compared  with  the  results  that  are  specified 
in  the  ASD  model,  namely  that  the  range  of  b2’s  is  narrow 
and  most  parameter  values  fall  between  1.10  and  1.0  (4:15). 

One  does  not  know  the  exact  value  of  the  exponent 
until  after  submission  of  the  final  CPR.  In  this  study,  the 
historical  parameters  for  similar  completed  aircraft  pro¬ 
grams  of  1.0,  1.06  and  1.10  are  used  as  subjective  values 
of  bg.  A  step  by  step  summary  of  the  computational  process 
for  generating  an  estimate  at  completion  is  as  follows: 

Method  6.  1.  Specify  the  value  of  b-  to  be  used 

b2 

m  the  functional  form  ACWP  =  b^  .  BCWP  .  Given  the  sample 
of  monthly  cumulative  values  available  to  date  for  ACWP  and 
BCWP  and  the  transformed  function  LOG  ACWP  =  LOG  b^  +  b2  LOG 
BCWP,  the  least  squares  technique  is  used  to  compute  the 
value  of  b^ .  Note:  b2  is  a  constant  in  this  method. 
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2.  Using  the  current  value  of  the  "budget  at  com¬ 
pletion  (BAC)  and  the  computed  value  of  b^  and  the  specified 

value  for  b ,,  an  estimate  at  completion  or  cost  at  comple- 

b2 

tion  is  calculated  using  the  function  EAC  =  b^  .  BAC 

3.  Steps  1  and  2  are  repeated  with  the  addition  of 
subsequent  monthly  CPR  data. 

The  preceding  sections  explained  the  procedure  and 
computational  formulas  used  in  the  various  methods  to  gen¬ 
erate  a  cost  at  completion.  In  review,  one  can  easily  rec¬ 
ognize  that  the  two  models  are  quite  diverse.  Having  exam¬ 
ined  the  computational  procedures  for  each  method,  the  next 
section  describes  measures  of  accuracy  required  in  testing 
for  differences  between  the  methods. 

Measures  of  Forecasting  Accuracy 

In  this  study  the  monthly  sample  data  and  methods  1 
through  6  are  used  to  generate  estimates  of  the  cost  at 
completion  ( Qj^p)  •  As  explained  earlier,  estimates  are  gen¬ 
erated  beginning  with  the  sixth  month  of  cost  performance 
data.  For  each  procedure  or  method,  a  separate  cost  at 
completion  estimate  is  generated  with  an  additional  month's 
cost  data  until  completion  of  the  program.  The  measure  of 
accuracy  is  defined  to  be  the  absolute  difference  between 
the  estimated  cost  at  completion  and  the  final  ACWP  value 
reported  in  the  last  CPR  expressed  as  a  percentage  of  final 
ACWP  (D^p).  For  each  method  by  contract,  the  mean  of  the 
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monthly  absolute  percentage  differences  is  calculated 
(%p) '  ^  summary  ^he  measures  of  accuracy  is  provided 

below  for  easy  reference: 

1.  Qj_-tp  =  estimate  of  CAC  for  procedure  i  generated 

in  month  t  for  program  p. 

2.  D-^  =  absolute  percentage  difference  of  Q.^ 

minus  the  final  ACWP  for  program  p. 


Ditp 


Qjtn  “  ACWPt 


ACWP_ 


3.  M-  =  mean  absolute  percentage  difference  for 

procedure  i  and  program  p.  It  is  the  sum 
of  divided  by  the  number  of  cost 

estimates  (n)  generated  with  procedure  i 
for  a  given  program. 

“ip  =  -r* 


Accuracy  can  be  measured  in  a  number  of  ways.  The  sum  of 
absolute  errors,  the  mean  square  error-  (MSE),  and  the  mean 
absolute  percentage  error  (MAPE)  are  but  a  few  measures.  The 
MSE  and  sum  of  error  measures  of  accuracy  are  absolute  meas¬ 
ures  of  the  magnitude  of  error.  The  disadvantage  is  that 
they  do  not  allow  for  comparison  across  treatments.  In¬ 
tuitively,  an  error  associated  with  a  large  dollar  value  pro¬ 
gram  can  easily  distort  comparisons  (10:8).  For  this  reason, 


32 


the  error  is  expressed  as  a  percentage  with  the  major  ad¬ 
vantage  being  that  the  treatments  can  be  compared. 

Testing  Procedure 

As  a  general  explanation,  each  of  the  procedures 
(Methods  1-6)  is  applied  to  the  sample  of  completed  programs. 
After  calculating  the  measures  of  accuracy  discussed  above, 
the  M's  are  tested  to  determine  whether  or  not  a  significant 
difference  exists  between  treatment  means.  A  single  factor 
analysis  of  variance  (ANQVA)  with  a  randomized  complete 
block  design  is  used  to  test  for  treatment  mean  equality. 

A  randomized  block  design  is  used  in  the  ANOVA  model  since 
differences  among  programs  contribute  to  the  variability 
observed  in  the  measure  of  accuracy.  As  a  result,  the  meas¬ 
ure  of  accuracy  reflects  both  random  error  and  variability 
between  programs.  A  design  to  remove  the  variability  be¬ 
tween  programs  is  to  test  each  method  on  each  program.  This 
strategy  improves  the  accuracy  of  the  comparisons  between 
methods.  Using  an  F  test  for  the  equality  of  treatment  means, 
the  alternative  conclusions  are; 

V  M-1  -  ^2  *  •  '  *  1^6 

H^:  Not  all  p ' s  are  equal 
The  decision  rule  to  control  the  a  risk  is: 

If  F*  <  F  ( 1 -a ;  r-1,  n^-r),  conclude  Hq 
If  F*  >  F  (1-a;  r-1,  n^-r),  conclude 
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where  F*  =  ^||§,  and  a  =  .05  (11*532-535) . 


If  Hq  is  rejected,  Fisher's  least  significant  difference 
(LSD)  test  is  applied  to  detect  the  true  differences  between 
treatment  means.  The  LSD  test  specifies  which  treatment 
means  are  significantly  different  from  others  (12:48-49). 
Having  discussed  the  general  testing  procedure,  the  test 
design  is  related  to  the  objectives  and  hypotheses  presented 
in  Chapter  1 . 

The  first  phase  of  testing  is  designed  to  address 
the  overall  accuracy  of  the  methods  irrespective  of  the  type 
of  program  being  estimated.  Methods  1-5  are  applied  to  the 
entire  sample  of  30  programs.  After  calculating  the  measures 
of  accuracy,  the  statistical  tests  discussed  above  are  ap¬ 
plied.  The  results  of  these  tests  indicate  whether  or  not 
one  method  is  preferable  overall  to  another.  It  should  be 
noted  that  method  6  is  not  run  against  the  entire  sample  of 
30  programs.  The  specified  values  of  b2  ranging  from  1.0  to 
1.10  are  applicable  to  aircraft  procurements  only.  Method  6 
is  evaluated  by  running  the  procedures  against  those  aircraft 
programs  included  in  the  sample.  Likewise,  analysis  of  the 
b2  parameter  is  based  on  aircraft  programs  in  the  sample. 

The  second  phase  of  testing  is  designed  to  answer 
the  second  the  third  hypotheses.  The  basic  question  ad¬ 
dressed  is  as  the  percentage  of  contract  completion  changes 
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is  one  method  preferable  to  another?  To  answer  this  ques¬ 
tion,  the  following  design  is  employed: 

1.  Determine  the  months  in  which  the  contractor  has 
performed  25,  50,  and  75  percent  of  the  contractual  effort. 

%  complete  (25,  50,  or  75)  =  ^BCWP 

.dAO 

2.  Calculate  D^^'s  in  same  manner  as  previously  de¬ 
scribed. 

3-  Group  the  D^^'s  into  four  classes  as  follows: 

a.  O^p's  generated  with  data  where  contract  is 

equal  to  or  less  than  25  percent  complete . 

%-fcp's  generated  with  data  where  contract  is 

greater  than  25  percent  complete  but  less  than 
50  percent  complete. 

c.  D^-tp's  generated  with  data  where  contract  is 

greater  than  50  percent  complete  but  less  than 
75  percent  complete. 

d.  D^^'s  generated  with  data  where  contract  is 
greater  than  75  percent  complete. 

4.  For  each  class,  is  calculated, however,  n  is 
replaced  by  the  number  of  D^^'s  for  a  program  included  in 
each  class. 

MOVA  and  the  LSD  test  are  applied  to  each  class.  For  method 
6,  three  variations  of  the  method  are  used  where  \>2  is  set 
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equal  to  1.0,  1.06,  or  1.10.  Additionally,  it  is  noted  that 
the  sample  for  this  phase  of  testing  consists  of  all  air¬ 
craft  programs  (20  programs). 

The  final  phase  of  testing  is  really  an  examination 
of  the  computed  b2  parameters  for  the  aircraft  programs  in 
the  sample.  The  ASD  model  hinges  on  the  premise  that  b2 
parameters  for  aircraft  programs  have  a  narrow  range  between 
.97  and  1.18,  and  in  the  majority  of  cases  b2  falls  between 
1.0  and  1.10  (4:15).  The  b2  values  computed  for  the  20 
aircraft  programs  are  analyzed  by  calculating  the  mean  b2 
value  and  sample  standard  deviation.  Following  the  test 
design  described  above,  Chapter  4  presents  the  test  results 
of  the  analysis. 
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Chapter  4 
RESULTS 

Chapter  4  presents  the  results  of  this  study.  Each 
hypothesis  is  restated  along  with  its  alternative  conclusions. 
The  pertinent  AN OVA  tables  are  provided  noting  the  signific¬ 
ance  of  the  F  test.  Where  a  significant  difference  between 
methods  is  indicated  by  ANOVA,  the  Fisher  least  significant 
difference  (LSD)  test  statistic  is  calculated.  As  a  visual 
representation,  a  number  line  is  used  to  portray  treatment 
mean  location  and  significant  differences. 

First  Hypothesis 

Are  the  exponential  methods  more  accurate  than  .the 
linear  case  variance  extrapolation  methods  in  forecasting  an 
estimate  at  completion?  The  measure  of  accuracy  is  a  monthly 
mean  absolute  percentage  error  (MAPE)  computed  and  averaged 
over  the  total  span  of  a  program.  The  alternative  conclu¬ 
sions  are: 

V  4i  =  42  =  • • •  = 

s  not  all  p '  s  are  equal 

where  p^,  p2 ,  p y  and  p^  are  the  means  of  the  distribu¬ 
tions  of  absolute  average  percentage  differences  for  the 
ESD  methods  previously  described,  p^  is  the  ASD  "unconstrained" 
method,  and  pg  is  the  ASD  "constrained"  method  with  b2  equal 


to  1.0 
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Table  1  is  a  single-factor  ANQVA  table  generated  by- 
running  the  above  methods  with  all  30  sample  programs.  The 
F  test  statistic  (1.173)  is  not  significant  at  a  =  .05* 
Therefore,  HQ  is  not  rejected  and  it  is  concluded  that  the 
distributions  of  the  average  percentage  differences  have  the 
same  mean  when  the  six  methods  are  applied  to  the  overall 
sample  of  30  programs. 


Table  1 

Single  Factor  ANOVA  Table*.  All  Programs 
Using  Six  Estimating  Methods 


Source  of 
Variation 

Sum  of 
Squares 

DF 

Mean 

Square 

F 

Significance 
of  F 

Method  of 
Forecasting 

240.214 

5 

48.043 

1.173 

0.325 

Blocking  for 

Individual  9175*857 
Programs 

30 

305.862 

Residual  6143.866 

150 

40.959 

Total  15,559.937 

185 

84.108 

The  linear  and  non-linear  methods  are  applied  to  the 
sample  of  20  aircraft  programs.  Again,  the  first  hypothesis 
is  addressed;  that  the  exponential  methods  are  more  accurate 
than  the  linear  cost  variance  extrapolation  methods  in  fore¬ 
casting  the  contract  cost  at  completion.  The  alternative 
conclusions  are  similars 

Hq:  4^  =  p_2  =  ■  •  •  =  4g 
H^s  not  all  p's  are  equal 
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where 


fig,  fir,,  and  fig  are  the  means  of  the  distributions  of 
average  percentage  differences  for  the  ASD  "constrained" 
method  specifying  b2  values  of  1.0,  1.06,  and  1.10,  respec¬ 
tively  . 

Table  2  is  a  single-factor , ANOVA  table  generated  by 
running  the  above  methods  with  the  sample  of  20  aircraft  pro¬ 
grams.  The  F  test  statistic  ( 5 • 97 8 )  is  significant  at  a  =  .05* 
Therefore,  HQ  is  rejected  and  it  is  concluded  that  not  all 
distributions  of  the  average  percentage  differences  have  the 
same  mean.  At  least  one  mean  is  significantly  different 
from  another.  Additional  information  is  provided  to  assess 
the  difference  in  treatment  means. 


Table  2 

Single-factor  ANOVA  Tables  Aircraft  Programs 
Using  Eight  Estimating  Methods 


Source  of 
Variation 

Sum  of 
Squares 

DF 

Mean 

Square 

F 

Significance 
of  F 

Method  of 
Forecasting 

615-301 

7 

87.900 

5-978 

0.000 

Blocking  for 
Individual 
Programs 

1286.728 

19 

67.723 

Residual 

1955-667 

.  .133 

14. 704 

Total 

3857.696 

159 

24.262 

Table  3  presents  the  mean  absolute  percentage  error 
for  each  treatment  or  method.  The  Fisher  LSD  test  statistic 
is  calculated  as  follows: 
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/  2  MSR 

v  v  — “ 


Fisher's  LSD  = 

where  v  =  degrees  of  freedom,  MSR  =  mean  square  re 
sidual,  and  n  =  the  sample  size. 

Given  a  =  .05,  v  =  7,  MSR  =  14.704,  and  n  =  20 
LSD  =  t  .025,  7  V 

=  2.571  V  1.^70 
=  3-18 


Table  3 

Mean  Absolute  Percentage  Error  For  Each 
Methods  All  Aircraft  Programs 

Mean  Absolute 


Forecasting  Method 

Percentage  Error 

1. 

Current  Month  P.F. 

6.88$ 

2. 

Moving  Average  P.F. 

5.06fo 

3. 

Cumulative  P.F. 

4.57# 

4. 

Quarterly  P.F. 

4.64# 

5. 

Unconstrained 

5.38# 

6. 

Constrained  (1.0) 

6 . 36# 

7. 

Constrained  ( 1 .06) 

10.77 # 

8. 

Constrained  (1.10) 

7.87# 
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X,  X 


3  4 


x2  x5 


4.57  4.64  5.06  5.38 


6.36  6.88  7.87  10.77 


*MAPE  ' s  within  brackets  are  not  significantly  different  at 
a  =  .05 


Figure  1.  Treatment  Mean  Locations:  Aircraft  Programs 


As  portrayed  in  Figure  3»  the  absolute  percentage 
error  for  Method  7  differs  significantly  from  the  means  for 
Methods  1,  2,  3,  4,  5*  and  6.  Method  8  differs  significantly 
from  Methods  3  and  4.  Since  Methods  7  and  8  are  non-linear, 
the  general  hypothesis  that  the  non-linear  model  is  more  ac¬ 
curate  than  the  linear  model  is  not  -upported. 

Second  Hypothesis 

The  next  hypothesis  states  that  the  exponential 
methods  are  more  accurate  than  linear  cost  variance  extra¬ 
polation  methods  in  forecasting  an  estimate  at  completion 
when; 

a.  the  contract  is  less  than  25$  complete 

b.  the  contract  is  greater  than  or  equal  to  25#  but 
less  than  50$  complete 

c.  the  contract  is  greater  than  or  equal  to  50$  but 


less  than  75$  complete. 


For  all  classes  the  alternative  conclusi jns  are  the  same: 

V  “  ^2  =  *  *  •  =  ^8 

:  not  all  n ' s  are  equal 

Table  4  is  a  single -factor  ANOVA  table  generated  by 
running  the  above  methods  with  12  aircraft  programs.  Only 
estimates  generated  while  the  programs  are  less  than  25$ 
complete  are  considered.  The  F  test  statistic  (1.226)  is 
not  significant  at  a  =  .05* 

Table  4 

Single-factor  ANOVA  Table:  Contract  Completion 
Less  than  25$  Complete 

Source  of  Sum  of  Mean  Significance 

Variation _ Squares  SF  Square  F of  F _ 

Method  of 

Forecasting  613.906  7  87*701  1.226  ..0.299  •*  *•* 

Blocking  for 

Individual  4018.715  11  365*338 

Programs 

Residual  5509*900  77  71.557 

Total  11,142.521  95  IO6.763 

Therefore,  hQ  is  not  rejected  and  it  is  con¬ 
cluded  that  the  distributions  of  the  average  percentage  errors 
have  the  same  mean.  The  non-linear  methods  are  not  more  ac¬ 
curate  at  forecasting  than  the  linear  methods  when  the  con¬ 
tract  is  less  than  25$  complete. 
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In  Table  5>  the  eight  estimating  methods  are  used  to 
generate  estimates  while  the  program  is  greater  than  or  equal 
to  2 5fo  but  less  than  50%  complete.  The  methods  are  applied 
to  19  aircraft  programs.  As  indicated  in  the  single  factor 
ANOVA  table,  the  F  test  statistic  (1.009)  is  not  significant 
at  a  =  .05*  Hq  is  not  rejected  and  it  is  concluded  that  the 
non-linear  model  is  not  more  accurate  at  forecasting  over 
the  2 5%>  to  50%  completion  range. 

Table  5 

Single -factor  ANOVA  Tables  Contract  Completion 
Greater  than  or  equal  to  25%>  and  less  than  50%> 


Source  of 

Sum  of 

Mean 

Significance 

Variation 

Squares 

DF  Square  F 

of  F 

Method  of 
Forecasting 

472.469 

7  67.496  1.009 

0.428 

Blocking  for 
Individual 
Programs 

4289.929 

18  238.329 

Residual 

8427.281 

126  66.883 

Total  13,189.676 

131  87.349 

Applying  the  eight  estimating  methods  to  19  aircraft 
programs  over  the  completion  range  greater  than  or  equal  to 
50%>  and  less  than  7 5%t  results  in  ANOVA  table  6.  As  indi¬ 
cated  in  the  table,  the  F  statistic  (3*490)  is  significant 
at  a  =  .05*  The  null  hypothesis  is  rejected,  allowing  the 
conclusion  that  not  all  distribution  means  are  equal. 
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Table  6 


Single-factor  ANOVA  Tables  Contract  Completion 
Greater  than  or  equal  to  50 %>  and  less  than  75% 

Source  of  Sum  of  Mean  Significance 

Varlati on _ Squares  DF  Square _ F _ of  F _ 


Method  of 
Forecasting 

585.147 

7 

83.592  3.490 

0.002 

Blocking  for 
Individual 

14.54,940 

18 

80.830 

Programs 

Residual 

3018.093 

126 

23.953 

Total 

5058.181 

151 

33.498 

Table  7  provides  the  MAPE  values  for  each  method. 
The  LSD  test  statistic  is  calculated  below: 


Fisher's  LSD  =  tcx/2,  v  V  2_JSR 

Given  a  =  .05»  v  =  7»  MSR  =  12.953»  and  n  =  19 


lsd  =  2.365  V 

19 


=  3.76 


Table  7 

Mean  Values  for  Each  Method:  $0%  to  75%  complete 


Mean  Absolute 


Forecasting  Method 

Percentage  Error 

1. 

Current  Month  P.F. 

8.637s 

2. 

Moving  Average  P.F. 

6.04?S 

3- 

Cumulative  P.F. 

5.3475 

4. 

Quarterly  P.F. 

5.217S 

5. 

Unconstrained 

6.947S 

6. 

Constrained  (1.0) 

7.447S 

7. 

Constrained  (1.06) 

8.807S 

8. 

Constrained  (1.10) 

11.44% 

Figure  2  visually  depicts  the  mean  absolute  percent- 

age 

error  by  method  on  a 

.  number  line.  Using  the  calculated 

LSD 

statistic,  the  mean 

of  Method  8  differs 

x. 


x. 


X1  x7 

A - 1- 


l8 


- 1 - ( - 1 - 1 - I- - 

5.21  5.34  6.04  6.94  7.44  8.63  8.80 

1 _ LSD  =  3.76 _ 


11 .44 


*  MAPE's  within  brackets  are  not  significantly  different  at 

a  =  .05. 

Figure  2.  Treatment  Mean  Locations:  50%  to  75%>  complete, 
from  the  means  of  Methods  2,  3,  4,  5,  and  6. 
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A  significant  difference  "between  treatment  means  exists  as 
indicated  by  ANOVA.  However,  since  Method  8  is  a  non-linear 
method  the  hypothesis  that  non-linear  methods  are  more  ac¬ 
curate  is  not  supported. 

Third  Hypothesis 

When  estimates  are  generated  for  a  program  which  is 
greater  than  or  equal  to  75%  complete,  a  linear  model  or 
non-linear  model  is  not  significantly  more  accurate  as  a 
program  approaches  completion.  The  alternative  conclusions 
remain  unchanged: 

V  ^1  =  ^2  =  *•*  =  ^8 

H^ :  not  all  b's  are  equal. 

For  Table  8,  the  eight  estimating  methods  are  applied 
to  generate  estimates  where  the  programs  are  equal  to  or 
greater  than  75%  complete.  Based  on  a  sample  size  of  19  ^ 

aircraft  programs,  the  F  statistic  (8,180)  indicates  that 
all  treatment  means  are  not  equal  at  a  =  .05*  The  Fisher 
LSD  test  statistic  is  calculated  as  follows: 

Fisher’s  LSD  =  ta/2,  v  J 


Given  a  =  .0 5.  v  =  7#  MSR  =  11. 589 »  and  n  =  19 


Table  8 


Single-factor  ANOVA  Table:  Contract  Completion 
Equal  to  or  Greater  Than  75$ 


Source  of 
Variation 

Sum  of 
Squares 

DF 

Mean 

Square 

F 

Significance 
of  F 

Method  of 
Forecasting 

663.614 

7 

94.802 

8.180 

0.000 

Blocking  for 
Individual 
Programs 

1269.141 

18 

70.508 

Residual 

l46o .193 

126 

ON 

cc 

XP| 

• 

ll 

Total 

3392.947 

151 

22.470 

Using  the  calculated  value  for  LSD  and  treatment 
mean  values  provided  in  Table  9*  it  is  determined  that  a  sig¬ 
nificant  difference  exists  between  Method  8  and  all  other 
Methods.  Method  7  differs  significantly  from  Methods  2,  3, 

4,  and  5-  These  results  are  portrayed  in  Figure  3.  The 
null  hypothesis  is  rejected  and  it  is  concluded  that  some 
methods  are  preferable  to  others  when  estimating  a  cost  at 
completion  even  when  the  contract  is  greater  than  75$  com¬ 
plete  . 

Table  9 

Mean  Values  for  each  Method:  Greater  than  75$  complete 

Mean  Absolute 

_ Forecasting  Method _ Percentage  Error _ 


1. 

Current  Month  P.F. 

5.49$ 

2. 

Moving  Average  P.F. 

4.48?S 

3- 

Cumulative  P.F. 

3.42?S 

4. 

Quarterly  P.F. 

4.05$ 
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Table  9  (Continued) 


Mean  Absolute 

Forecasting  Method _ Percentage  Error 


5. 

Unconstrained 

4.54?S 

6 . 

Constrained 

(1.0) 

5.61% 

7. 

Constrained 

(1.06) 

7-51% 

8. 

Constrained 

(1.10) 

10.26 <% 

0 

*3  x^  x2  x^ 

|  |  |  | 

'*1  *6 

*7  *8 

3.42  4.05  4.48  4.54 

5.49  5-61 

7-51  10.26 

1 

* 

L 


*  MAPE's  Within  brackets  are  not  significantly  different  at 
a  =  .05. 

Figure  3.  Treatment  Mean  Locations:  Greater  than 
75%>  complete. 

Fourth  Hypothesis 

The  fourth  hypothesis  is  really  an  examination  of  the 
parameter  b,,.  Use  of  the  "constrained"  ASD  method  hinges  on 
the  distribution  of  b2  which  was  noted  to  have  a  range  be¬ 
tween  .97  and  1.18  and  in  the  majority  of  cases  b2  falls  be¬ 
tween  1.0  and  1.10.  Appendix  B  contains  the  calculated  b2 
parameters  for  the  sample  of  20  aircraft  programs.  The 
range  for  the  sample  b2  values  is  .9098  to  1.1209.  The 


mean  sample  b 2  value  is  1.0338  with  a  standard  deviation  and 
variance  of  .0564  and  .0030,  respectively.  Based  on  the 
observations  of  bg  and  assuming  the  sample  is  normally  dis¬ 
tributed,  a  95ft  confidence  interval  has  upper  and  lower  limits 
of  .9232  and  1.144,  respectively.  Fourteen  of  the  20  sample 
program  b^  parameters  fall  within  the  stated  range  of  1.0 
to  1.10.  The  sample  observation  values  of  bg  support  the 
assertion  that  bg  in  the  majority  of  cases  falls  between 
1 .0  and  1.10. 
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Chapter  5 


ANALYSIS  AND  RECOMMENDATIONS 

Summary  of  the  Research  Effort 

In  Chapter  1  the  hypotheses  under  investigation  are 
presented.  In  general  the  hypotheses  state  that  the  non¬ 
linear  estimating  model  is  preferable  to  the  linear  model  in 
estimating  a  contract  cost  at  completion.  A  measure  of  ac¬ 
curacy  mean  absolute  percentage  error  (MAPE)  was  defined 
equal  to  the  estimated  value  minus  the  actual  final  cost  at 
completion  divided  by  the  actual  final  cost.  The  six  esti¬ 
mating  methods  described  in  Chapter  3  were  first  applied  to 
a  sample  of  30  programs.  ANQVA  results  did  not  indicate  a 
significant  difference  among  treatment  means.  Eight  esti¬ 
mating  methods,'  three  of  which  used  specified  values  for  b2 
of  1.0,  1.06,  and  1.10,  were  applied  to  a  sample  of  20  air¬ 
craft  programs.  ANOVA  indicated  at  least  one  treatment  mean 
was  significantly  different  from  another.  Using  Fisher's  LSD 
test  statistic  it  was  determined  that  a  non-linear  method  had 
a  higher  treatment  mean  which  was  significantly  different. 

The  result  is  contrary  to  the  general  hypothesis  and  is  dis¬ 
cussed  further  below. 
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Review  of  Findings 

The  ax-ray  of  estimates  for  each  program  and  method 
v/ as  divided  into  four  classes  where  the  program  is  less  than 
i'5%  complete,  greater  than  for  equal  to  25%  but  less  than 
50%>  complete,  greater  than  or  equal  to  50 %  but  3. ess  than  75% 
complete,  and  greater  than  or  equal  to  75%  complete.  ANOVA 
did  not  indicate  a  significant  difference  among  treatment 
means  for  either  of  the  first  two  classes.  ANOVA  indicated 
inequality  of  treatment  means  in  the  third  class.  Two  non¬ 
linear  methods  had  MAPE's  which  were  significantly  higher 
than  linear  MAPE's.  The  same  pattern  held  for  the  fourth 
class.  Two  non-linear  treatment  means  were  significantly 
higher.  Again,  these  results  are  contrary  to  the  general 
hypothesis . 

•  Conclusions  -and  -Implications 

The  higher  mean  absolute  percentage  ex-rors  for  Methods 
7  and  8  may  be  a  result  of  specifying  too  high  a  value  for 
in  the  "constrained"  method.  For  the  sample  of  20  air¬ 
craft  programs,  the  mean  bg  parameter  value  is  1.033  not 

1.06  or  1.10  as  specified  in  Methods  7  and  8,  respectively. 

b? 

When  used  in  the  functional  form  y  =  b^  x  ,  an  estimate  is 
calculated  by  substituting  BAC  for  x.  The  significance  of 
this  point  can  be  highlighted  by  considering  a  simple  exam¬ 
ple  where  BAC  equals  1  million,  b^  equals  a  constant  (1.0), 
and  bg  equals  1.03,  1.06,  or  1.10.  If  bg  equals  1.03*  y 
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equals  1.51  million.  If  b2  equals  1.06  or  1.10,  y  equals 
2.29  or  3*98  million,  respectively.  Specifying  a  value  or 
range  of  values  for  b2  is  a  crucial  step  in  applying  the  non¬ 
linear  model. 

Another  special  case  to  consider  with  regard  to  the 
"constrained"  estimating  method  is  the  case  where  b2  is  set 
equal  to  1.0.  When  b2  equals  1.0,  the  estimating  relation¬ 
ship  reduces  to  y  =  b^,  BAC  where  b^  is  in  essence  a  perform¬ 
ance  factor.  In  this  form,  the  b^  parameter  is  multiplied  by 
the  aggregate  budget  at  completion.  The  ESD  methods  still 
differ  fundamentally  in  their  construction.  In  the  ESD 
methods,  the  performance  factor  is  multiplied  by  the  budgeted 
cost  of  work  remaining  (BCWR)  and  not  the  aggregate  budget 
term.  BCWR  is  calculated  by  subtracting  cumulative  BCWP 
from  BAC.  As  described  in  Chapter  3>  the  performance  factor 
used  in  the  ESD  model-  can  be  calculated  in  a  number  of  ways 
using  current  monthly  data  or  cumulative  data. 

Areas  for  Future  Research 

As  indicated  above,  the  observed  range  of  b2  values 
if  used  with  the  "constrained"  method  would  not  improve  the 
accuracy  of  the  model.  To  improve  accuracy  a  narrow  range 
needs  to  be  specified.  As  a  suggestion  for  further  research, 
a  detailed  analysis  of  aircraft  programs  by  type  of  aircraft 
may  help  to  narrow  or  specify  a  value  for  b2>  Additionally, 
a  detailed  analysis  may  indicate  that  b2  varies  depending 
whether  or  not  the  contract  is  a  follow-on  production  option. 
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This  analysis  compared  but  two  of  a  number  of  models 
for  estimating  a  contract  cost  at  completion.  Other  methods 
should  be  evaluated  particularly  those  mentioned  in  Chapter 
2.  It  is  important  to  note  that  any  model  represents  a 
mathematical  manipulation  of  the  information  available. 
Technical  assessment  of  a  contractor's  performance  provided 
by  a  SPO's  engineering  personnel  is  just  as  valuable  as  a 
performance  factor  calculated  on  past  cost  performance.  Fur¬ 
ther  it  should  be  recognized  that  the  models  discussed  use 
contractor  provided  cost  data.  Inaccurate  input  invalidates 
the  usefulness  of  the  models . 

Concluding  Remarks 

In  summaryi  the  results  obtained  in  this  analysis  do 
not  support  the  general  hypothesis  that  a  non-linear  model 
is  more  accurate  at  forecasting  an  estimate  at  completion 
than  a  linear  model.  The  ESD  linear  methods  were  found  to 
be  just  as  accurate  as  the  ASD  methods.  This  analysis  does 
not  content  that  either  model  be  abandoned  but  rather  that 
the  models  can  be  used  in  conjunction  with  each  other. 
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APPENDIX  A. 

SAMPLE  PROGRAM  WORKSHEET 


PROGRAM  TITLE 


ASD  FILE  NUMBER 

CUMULATIVE 

MONTH  BOWS  BCWP  ACWP  BAC  LRE 

1  XX.XX  XXXX  XX  XX  xxxx  xxxx 

2 

3 


n 
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APPENDIX  B 

b2  PARAMETER  VALUES  FOR  20  AIRCRAFT  PROGRAMS 
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